Skip to content

perf(web): resume syntax highlighting from completed lines - #11196

Merged
juliusmarminge merged 1 commit into
legend-perf/incremental-markdownfrom
legend-perf/streaming-highlighting
Sep 11, 2026
Merged

perf(web): resume syntax highlighting from completed lines#11196
juliusmarminge merged 1 commit into
legend-perf/incremental-markdownfrom
legend-perf/streaming-highlighting

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 11, 2026

Copy link
Copy Markdown
Member

Growing code fences tokenize every earlier line on each streamed chunk. Resume Shiki from the grammar state after the last completed line and tokenize the changing suffix. This adapts the incremental document work in Legend Code using Shiki's grammar-state API.

Completed messages keep the existing cached path. Non-append edits invalidate the prefix; CR/CRLF, plain text, ANSI and missing grammar state fall back to full highlighting. Multiline syntax and embedded languages retain their grammar context.

Five alternating production-browser runs at 1440 × 1000, each with 10 warm-up updates and 40 measured updates. The fixture starts with 200 TypeScript lines and appends one line per update.

Synchronous client update Base, including parser layer This layer
Median of run medians 170.8 ms 120.3 ms
Median of run p95s 211.5 ms 152.6 ms

The median reduction is 29.6%; every paired run improved. This is a deliberately heavy client-rendering workload, not provider throughput or typical-user latency. Probe and samples. The codeToHtmlMs counter excludes direct codeToHast work; the complete synchronous update is the comparison metric.

100 focused Markdown/highlighting tests and web typecheck pass. Actual WASM highlighting matches full output through streaming prefixes across 11 languages and two themes, including multiline comments, strings, heredocs, embedded HTML, edits, truncation and fallback languages. Targeted lint reports two warnings in unchanged code.

This improves streaming code in web and Electron's shared renderer. It does not change mobile, file/diff viewers, the wire protocol or provider adapters. Both hosted and locally served web clients use this code; no transport timing claim is made.

Before:

Before: highlighted TypeScript at the reading position

After:

After: matching highlighted TypeScript and layout

Before streaming:

https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/2fd3e3038ca91da9/highlight-before-realtime.webm

After streaming:

https://gh-file-drop-api-prod-mi5fy3sowv63ufte.pinglabs.workers.dev/f/cd992d3d57a131d6/highlight-after-realtime.webm

The videos reconstruct Chrome screenshot frames with their original timestamps, retaining gaps between paints. They are separate from the unrecorded timing runs. The same synthetic fixture and viewport are used; no private transcripts or credentials are included. No observed new blank frames or color flashes during the exercised stream.

Integration follow-up: rebased on current main; the integrated stack passes 94 focused Markdown/highlighting tests and 159 timeline tests, plus web typecheck. The timing table remains pinned to the original measured builds.

Model: GPT-6. Harness: Codex.

@juliusmarminge
juliusmarminge added this pull request to stack #11194 September 11, 2026 05:13
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Sep 11, 2026
@juliusmarminge
juliusmarminge marked this pull request as ready for review September 11, 2026 05:35
@macroscopeapp

macroscopeapp Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR replaces full streaming syntax highlighting with a stateful grammar/HAST-resumption path in the production chat renderer. Tests cover many languages, themes, edits, and fallbacks, but the runtime algorithm is sufficiently nontrivial that its streaming behavior warrants human review.

No code changes detected at ba2471b. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarminge force-pushed the legend-perf/streaming-highlighting branch from 6ab8db7 to ba2471b Compare September 11, 2026 05:44
@github-actions

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.6 KiB 15.1 KiB
Codex Thread snapshot wire 7.0 KiB 7.3 KiB
Codex Live turn WebSocket wire 6.6 KiB 7.8 KiB
Codex Live turn WebSocket decoded 57.1 KiB 66.4 KiB
Codex Live turn messages 10 21
Claude Total thread wire 13.6 KiB 15.1 KiB
Claude Thread snapshot wire 7.1 KiB 7.3 KiB
Claude Live turn WebSocket wire 6.5 KiB 7.8 KiB
Claude Live turn WebSocket decoded 57.8 KiB 66.4 KiB
Claude Live turn messages 9 21

Baseline: unavailable · PR result: ba2471b · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.9 KiB
  • Claude decoded thread snapshot: 114.6 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 5c212aaa-db8b-4400-8ed0-2430509ef6c0

📥 Commits

Reviewing files that changed from the base of the PR and between a660ad9 and ba2471b.

📒 Files selected for processing (3)
  • apps/web/src/components/ChatMarkdown.tsx
  • apps/web/src/lib/incrementalHighlighting.test.ts
  • apps/web/src/lib/incrementalHighlighting.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The change adds incremental syntax highlighting for streaming code blocks. It preserves grammar state across complete lines, falls back to full highlighting when needed, and adds tests for languages, themes, edits, truncation, plaintext, ANSI-like input, and line endings.

Changes

Incremental syntax highlighting

Layer / File(s) Summary
Incremental highlighter implementation
apps/web/src/lib/incrementalHighlighting.ts
createIncrementalHighlighter caches grammar state, reuses highlighted nodes, invalidates changed prefixes, and applies full-highlighter fallbacks.
Incremental highlighter validation
apps/web/src/lib/incrementalHighlighting.test.ts
Tests compare incremental output with full highlighting across languages, themes, edits, truncation, plaintext, ANSI-like input, and partial CRLF or CR endings.
Streaming code-block integration
apps/web/src/components/ChatMarkdown.tsx
Streaming blocks use the incremental highlighter. Non-streaming blocks continue to use full-document highlighting.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant UncachedShikiCodeBlock
  participant createIncrementalHighlighter
  participant DiffsHighlighter
  UncachedShikiCodeBlock->>createIncrementalHighlighter: Highlight streaming content
  createIncrementalHighlighter->>DiffsHighlighter: Highlight uncached complete lines
  DiffsHighlighter-->>createIncrementalHighlighter: Return highlighted nodes and grammar state
  createIncrementalHighlighter-->>UncachedShikiCodeBlock: Return incremental HTML
Loading

Merge Risk: ⚪ Minimal · up to ba247

The streaming highlighting change has no confirmed current-head issue requiring resolution before merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: resuming syntax highlighting from completed lines during streaming.
Description check ✅ Passed The description explains the change, motivation, implementation scope, performance results, testing, and UI evidence. It does not use the template headings or include the checklist, but it is mostly c…
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch legend-perf/streaming-highlighting

Comment @coderabbitai help to get the list of available commands.

@juliusmarminge
juliusmarminge merged commit d7d7f8f into main Sep 11, 2026
24 checks passed
@juliusmarminge
juliusmarminge deleted the legend-perf/streaming-highlighting branch September 11, 2026 06:50
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Sep 11, 2026
## What's Changed
* fix(pr): update labels and reviewers without redundant reloads by @maria-rcks in pingdotgg/t3code#11117
* fix(chat): fold question answers into tool activity by @maria-rcks in pingdotgg/t3code#11014
* fix(usage): flag unpriced model activity instead of showing $0.00 by @maria-rcks in pingdotgg/t3code#11021
* fix(server): let Claude launch args override the derived permission mode by @maria-rcks in pingdotgg/t3code#11026
* fix(editors): accept root paths and Windows servers in Zed remote links by @maria-rcks in pingdotgg/t3code#11044
* fix(web): center pull request unavailable states by @maria-rcks in pingdotgg/t3code#11110
* fix(web): remove sidebar pull request link icon by @maria-rcks in pingdotgg/t3code#11179
* fix(ui): color linked pr counts by aggregate status by @maria-rcks in pingdotgg/t3code#11180
* fix(preview): render website favicons for browser tool activity by @maria-rcks in pingdotgg/t3code#11032
* fix(web): simplify pull request summary sections by @maria-rcks in pingdotgg/t3code#10612
* fix(web): preserve drafts when compacting context by @maria-rcks in pingdotgg/t3code#11103
* fix(server): queue messages during context compaction by @maria-rcks in pingdotgg/t3code#11107
* perf(web): format minimap previews only when opened by @juliusmarminge in pingdotgg/t3code#11181
* perf(web): reuse completed Markdown prefixes while streaming by @juliusmarminge in pingdotgg/t3code#11193
* perf(web): resume syntax highlighting from completed lines by @juliusmarminge in pingdotgg/t3code#11196
* perf(web): preserve completed code-line DOM while streaming by @juliusmarminge in pingdotgg/t3code#11198
* perf(web): huge-thread switch no longer blanks the chat pane by @juliusmarminge in pingdotgg/t3code#11169


**Full Changelog**: pingdotgg/t3code@v0.0.41-nightly.20260911.1520...v0.0.41-nightly.20260911.1533

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.41-nightly.20260911.1533
AIdoesmyjob pushed a commit to AIdoesmyjob/t3code that referenced this pull request Sep 11, 2026
aorwall added a commit to aorwall/t3code that referenced this pull request Sep 12, 2026
Merges `upstream/main` at `e81606494` into the fork, from merge base
`02297e3db` — 47 upstream commits.

The theme of this range is scopable settings: upstream made every server
setting addressable at a scope (global / environment / project) with
per-project overrides, which is why 11 of the 15 conflicts are settings
files. The rest is conversation rewind, floating device streams, and a
large batch of message-sync and markdown-streaming perf work.

## Merge stats

- Landed (`HEAD^1..HEAD`): 277 files, 17243+/4783−
- Upstream range (base..`HEAD^2`): 275 files, 17011+/4749−
- Fork delta (`HEAD^2..HEAD`): 756 files, 76559+/2096−

The two file lists reconcile: the 3 extra landed files are
`docs/fork/inventory.json`, `docs/fork/upstream-merge-log.md` and
`docs/fork/gaps.md`; the 1 file in the range that did not land is
`apps/web/src/routes/settings.integrations.tsx`, resolved `ours` per the
`moatless-admin-integrations-route` inventory entry (that route is a
Moatless admin page here, and upstream's embedded-surface settings live
at `/settings/browser`).

All 15 conflicts were resolved by the verdict `preflight.mjs` printed.
No `decide` conflict was left unresolved. Details, including the
owned-concern sweep (no keyword hits) and the unsupported-method
reconciliation (0 ADD, 0 DROP, 2 KEEP, 4 known exceptions), are in the
dated entry in `docs/fork/upstream-merge-log.md`.

Two findings worth naming here:

- **A silent auto-merge failure.** pingdotgg#11285 changed the mini-player target
from a tab id to a source union. Git updated upstream's own assertion in
`PreviewView.test.tsx` and left the fork-only "under the frame
capability" case next to it still asserting the old string. No conflict
marker, no `resolution-check.mjs` finding — only the fork's own test
suite caught it.
- **Stale inventory anchors.** Upstream moved the project Actions
section out of `ProjectSettingsPanel.tsx` into a new
`ProjectActionsSettings.tsx`, which is where `scriptsEditable` is now
derived and where upstream's new writing Reset button is gated. Four
inventory entries were re-pointed in this merge rather than silently
dropping their deltas.

## Usable as-is

Client work the fork can expose with no Moatless backend change:

- Scoped settings UI and the two-select scope picker (pingdotgg#10639, pingdotgg#10636) —
`SettingsScopeContext`, `ScopedSwitch`, `settingKeys`, the `mixed`
state. The reading half works against Moatless today.
- Float device streams over chat, as a source union rather than a tab id
(pingdotgg#11285); recording status on floating previews (pingdotgg#11312); floating
preview using composer margins (pingdotgg#11290).
- PR-page selections into new drafts (pingdotgg#11296);
projects-on-another-machine badge (pingdotgg#11323); Usage opening on Limits
(pingdotgg#11261).
- macOS permission onboarding (pingdotgg#11289); hold-to-quit fix (pingdotgg#11016);
preview keystrokes kept out of the composer (pingdotgg#11354).
- Message-sync and markdown-streaming perf: pingdotgg#11302, pingdotgg#11029, pingdotgg#11211,
pingdotgg#11198, pingdotgg#11196, pingdotgg#11193, pingdotgg#11181, pingdotgg#11206.
- Assorted web/mobile fixes: pingdotgg#11361, pingdotgg#10757, pingdotgg#11357, pingdotgg#10571, pingdotgg#11348,
pingdotgg#11349, pingdotgg#11281, pingdotgg#11188, pingdotgg#11283, pingdotgg#11292, pingdotgg#11187, pingdotgg#11228, pingdotgg#11103, pingdotgg#10612,
pingdotgg#11032, pingdotgg#11233, pingdotgg#11234, pingdotgg#11304, pingdotgg#11240.

## Unsupported in Moatless / needs implementation

- **Conversation rewind** — `thread.conversation.revert` (pingdotgg#11358). A new
member of `DispatchableClientOrchestrationCommand` in
`packages/contracts/src/orchestration.ts`, bringing the fork to 30
command types (28 upstream's, 2 fork-only). Moatless does not dispatch
it, and a client command cannot be refused per-type, so "Edit from here"
on `RevertUserMessageButton` is reachable whenever the turn is idle and
does nothing. Needs backend dispatch.
- **Per-project setting overrides** — the `projectSettingsOverrides`
capability and the 17-key `ProjectSettingsOverrides` record (pingdotgg#11176).
Two pieces are needed: the capability reported by
`/.well-known/t3/environment`, and `server.updateSettings` served at
project scope. Until both land, the capability filter in
`scopedSettings.ts:170` and `ProjectActionsSettings.tsx:72` drops the
write on the client — the control renders, the user toggles it, and
**the write never leaves the browser**. A silent no-op is worse than a
hidden control or an honest refusal; recorded in `docs/fork/gaps.md`.
- **Default thread permissions** — `defaultRuntimeMode` (pingdotgg#11346). Reads
fine, cannot be saved. Same `server.updateSettings` write path as above,
one level deeper, not a separate gap.

## Backend behavior to consider reproducing in Moatless

Upstream server-side work the fork cannot use directly, but that
Moatless would benefit from:

- **Queue messages during context compaction** (pingdotgg#11107,
`ProviderCommandReactor.ts`) — a message sent while compaction is in
flight is currently dropped rather than held.
- **Restore provider history and prompts when rewinding** (pingdotgg#11338,
`CheckpointReactor.ts`) — the counterpart to
`thread.conversation.revert` above; rewinding the thread without
rewinding provider state leaves the two out of sync.
- **Detect file renames in review diffs** (pingdotgg#8086,
`apps/server/src/vcs/GitVcsDriverCore.ts`) — a rename currently reads as
a whole-file delete plus a whole-file add.
- **Preserve qualified Codex model ids** (pingdotgg#9921, `ModelManifest.ts` +
`CodexTextGeneration.ts`).
- **Model defaults** astra-medium / fable-5.1-medium (pingdotgg#11347).

All five are recorded under the runtime-fixes entry in
`docs/fork/gaps.md`.

## Verification

`verify.mjs` (full pass): 7 of 8 checks green — `duplicate-adds`,
`tripwires`, `resolution-check`, `unsupported-methods`, `fmt:check`,
`lint`, `typecheck`.

`test` is red on **`@t3tools/desktop` only**, at
`scripts/browser-secret-native.test.mjs > bundled libsecret helper`:
`Command failed: pkg-config --cflags --libs libsecret-1`. This is the
standing sandbox gap, not a merge regression — the test file's last
commit is `498ab9c39` (pingdotgg#7261, before the merge base), `git diff
--name-only` against both merge parents is empty for it, and `pkg-config
--exists libsecret-1` fails in this environment. It is already an entry
in `docs/fork/gaps.md`. Every other package passes, including
`@t3tools/web` (5079 tests) after the `PreviewView.test.tsx` fix above.

Three typecheck failures the merge introduced were fixed in it:
`SETTINGS_CATEGORY_SCOPES` in `settingsSearch.ts` was missing all 9
fork-only settings paths, and two `filterAvailableSettingsSearchItems`
literals in `settingsSearch.test.ts` were missing the fork's
`forgejoEnabled` field.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---
Moatless task:
https://moatless.soaplabstest.com/tasks/e70b41b3-779d-43b8-8f34-7de516548e7c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant